Deep learning for prediction of colorectal cancer outcome: a discovery and validation study

Background Improved markers of prognosis are needed to stratify patients with early-stage colorectal cancer to refine selection of adjuvant therapy. The aim of the present study was to develop a biomarker of patient outcome after primary colorectal cancer resection by directly analysing scanned conventional haematoxylin and eosin stained sections using deep learning. Methods More than 12 000 000 image tiles from patients with a distinctly good or poor disease outcome from four cohorts were used to train a total of ten convolutional neural networks, purpose-built for classifying supersized heterogeneous images. A prognostic biomarker integrating the ten networks was determined using patients with a non-distinct outcome. The marker was tested on 920 patients with slides prepared in the UK, and then independently validated according to a predefined protocol in 1122 patients treated with single-agent capecitabine using slides prepared in Norway. All cohorts included only patients with resectable tumours, and a formalin-fixed, paraffin-embedded tumour tissue block available for analysis. The primary outcome was cancer-specific survival. Findings 828 patients from four cohorts had a distinct outcome and were used as a training cohort to obtain clear ground truth. 1645 patients had a non-distinct outcome and were used for tuning. The biomarker provided a hazard ratio for poor versus good prognosis of 3·84 (95% CI 2·72–5·43; p<0·0001) in the primary analysis of the validation cohort, and 3·04 (2·07–4·47; p<0·0001) after adjusting for established prognostic markers significant in univariable analyses of the same cohort, which were pN stage, pT stage, lymphatic invasion, and venous vascular invasion. Interpretation A clinically useful prognostic marker was developed using deep learning allied to digital scanning of conventional haematoxylin and eosin stained tumour tissue sections. The assay has been extensively evaluated in large, independent patient populations, correlates with and outperforms established molecular and morphological prognostic markers, and gives consistent results across tumour and nodal stage. The biomarker stratified stage II and III patients into sufficiently distinct prognostic groups that potentially could be used to guide selection of adjuvant treatment by avoiding therapy in very low risk groups and identifying patients who would benefit from more intensive treatment regimes

Chromatin organisation and cancer prognosis: a pan-cancer study

Background: Chromatin organisation affects gene expression and regional mutation frequencies and contributes to carcinogenesis. Aberrant organisation of DNA has been correlated with cancer prognosis in analyses of the chromatin component of tumour cell nuclei using image texture analysis. As yet, the methodology has not been sufficiently validated to permit its clinical application. We aimed to define and validate a novel prognostic biomarker for the automatic detection of heterogeneous chromatin organisation. Methods Machine learning algorithms analysed the chromatin organisation in 461 000 images of tumour cell nuclei stained for DNA from 390 patients (discovery cohort) treated for stage I or II colorectal cancer at the Aker University Hospital (Oslo, Norway). The resulting marker of chromatin heterogeneity, termed Nucleotyping, was subsequently independently validated in six patient cohorts: 442 patients with stage I or II colorectal cancer in the Gloucester Colorectal Cancer Study (UK); 391 patients with stage II colorectal cancer in the QUASAR 2 trial; 246 patients with stage I ovarian carcinoma; 354 patients with uterine sarcoma; 307 patients with prostate carcinoma; and 791 patients with endometrial carcinoma. The primary outcome was cancer-specific survival. Findings: In all patient cohorts, patients with chromatin heterogeneous tumours had worse cancer-specific survival than patients with chromatin homogeneous tumours (univariable analysis hazard ratio [HR] 1·7, 95% CI 1·2–2·5, in the discovery cohort; 1·8, 1·0–3·0, in the Gloucester validation cohort; 2·2, 1·1–4·5, in the QUASAR 2 validation cohort; 3·1, 1·9–5·0, in the ovarian carcinoma cohort; 2·5, 1·8–3·4, in the uterine sarcoma cohort; 2·3, 1·2–4·6, in the prostate carcinoma cohort; and 4·3, 2·8–6·8, in the endometrial carcinoma cohort). After adjusting for established prognostic patient characteristics in multivariable analyses, Nucleotyping was prognostic in all cohorts except for the prostate carcinoma cohort (HR 1·7, 95% CI 1·1–2·5, in the discovery cohort; 1·9, 1·1–3·2, in the Gloucester validation cohort; 2·6, 1·2–5·6, in the QUASAR 2 cohort; 1·8, 1·1–3·0, for ovarian carcinoma; 1·6, 1·0–2·4, for uterine sarcoma; 1·43, 0·68–2·99, for prostate carcinoma; and 1·9, 1·1–3·1, for endometrial carcinoma). Chromatin heterogeneity was a significant predictor of cancer-specific survival in microsatellite unstable (HR 2·9, 95% CI 1·0–8·4) and microsatellite stable (1·8, 1·2–2·7) stage II colorectal cancer, but microsatellite instability was not a significant predictor of outcome in chromatin homogeneous (1·3, 0·7–2·4) or chromatin heterogeneous (0·8, 0·3–2·0) stage II colorectal cancer. Interpretation: The consistent prognostic prediction of Nucleotyping in different biological and technical circumstances suggests that the marker of chromatin heterogeneity can be reliably assessed in routine clinical practice and could be used to objectively assist decision making in a range of clinical settings. An immediate application would be to identify high-risk patients with stage II colorectal cancer who might have greater absolute benefit from adjuvant chemotherapy. Clinical trials are warranted to evaluate the survival benefit and cost-effectiveness of using Nucleotyping to guide treatment decisions in multiple clinical settings

Prognostic Value of Adaptive Textural Features–The Effect of Standardizing Nuclear First-Order Gray Level Statistics and Mixing Information from Nuclei Having Different Area

Background: Nuclear texture analysis is a useful method to obtain quantitative information for use in prognosis of cancer. The first-order gray level statistics of a digitized light microscopic nuclear image may be influenced by variations in the image input conditions. Therefore, we have previously standardized the nuclear gray level mean value and standard deviation. However, there is a clear relation between nuclear DNA content, area, first-order statistics, and texture. For nuclei with approximately the same DNA content, the mean gray level increases with an increasing nuclear area. The aims of the present methodical work were to study: (1) whether the prognostic value of adaptive textural features varies with nuclear area, and (2) the effect of standardizing nuclear first-order statistics. Methods: Nuclei from 134 cases of ovarian cancer were grouped into intervals according to nuclear area. Adaptive features were extracted from two different image sets, i.e., standardized and non-standardized nuclear images. Results: The prognostic value of adaptive textural features varied strongly with nuclear area. A standardization of the first-order statistics significantly reduced this prognostic information. Several single features discriminated the two classes of cancer with a correct classification rate of 70%. Conclusion: Nuclei having an area between 2000–4999 pixels contained most of the class distance information between the good and poor prognosis classes of cancer. By considering the relation between nuclear area and texture, we avoided a loss of information caused by standardizing the first-order statistics and mixing data from cells having different nuclear area

The Use of Fractal Features from the Periphery of Cell Nuclei as a Classification Tool

A polygonization‐based method is used to estimate the fractal dimension and several new scalar lacunarity features from digitized transmission electron micrographs (TEM) of mouse liver cell nuclei. The fractal features have been estimated in different segments of 1D curves obtained by scanning the 2D cell nuclei in a spiral‐like fashion called “peel‐off scanning”. This is a venue to separate estimates of fractal features in the center and periphery of a cell nucleus. Our aim was to see if a small set of fractal features could discriminate between samples from normal liver, hyperplastic nodules and hepatocellular carcinomas. The Bhattacharyya distance was used to evaluate the features. Bayesian classification with pooled covariance matrix and equal prior probabilities was used as the rule for classification. Several single fractal features estimated from the periphery of the cell nuclei discriminated samples from the hyperplastic nodules and hepatocellular carcinomas from normal ones. The outer 25–30% of the cell nuclei contained important texture information about the differences between the classes. The polygonization‐based method was also used as an analysis tool to relate the differences between the classes to differences in the chromatin structure

The use of fractal features from the periphery of cell nuclei as a classification tool, Analytical Cellular Pathology 19

A polygonization-based method is used to estimate the fractal dimension and several new scalar lacunarity features from digitized transmission electron micrographs (TEM) of mouse liver cell nuclei. The fractal features have been estimated in different segments of 1D curves obtained by scanning the 2D cell nuclei in a spiral-like fashion called &quot;peel-off scanning&quot;. This is a venue to separate estimates of fractal features in the center and periphery of a cell nucleus. Our aim was to see if a small set of fractal features could discriminate between samples from normal liver, hyperplastic nodules and hepatocellular carcinomas. The Bhattacharyya distance was used to evaluate the features. Bayesian classification with pooled covariance matrix and equal prior probabilities was used as the rule for classification. Several single fractal features estimated from the periphery of the cell nuclei discriminated samples from the hyperplastic nodules and hepatocellular carcinomas from normal ones. The outer 25-30% of the cell nuclei contained important texture information about the differences between the classes. The polygonization-based method was also used as an analysis tool to relate the differences between the classes to differences in the chromatin structure

Prognostic Classification of Early Ovarian Cancer Based on very Low Dimensionality Adaptive Texture Feature Vectors from Cell Nuclei from Monolayers and Histological Sections

In order to study the prognostic value of quantifying the chromatin structure of cell nuclei from patients with early ovarian cancer, low dimensionality adaptive fractal and Gray Level Cooccurrence Matrix texture feature vectors were extracted from nuclei images of monolayers and histological sections. Each light microscopy nucleus image was divided into a peripheral and a central part, representing 30% and 70% of the total area of the nucleus, respectively. Textural features were then extracted from the peripheral and central parts of the nuclei images. The adaptive feature extraction was based on Class Difference Matrices and Class Distance Matrices. These matrices were useful to illustrate the difference in chromatin texture between the good and bad prognosis classes of ovarian samples. Class Difference and Distance Matrices also clearly illustrated the difference in texture between the peripheral and central parts of cell nuclei. Both when working with nuclei images from monolayers and from histological sections it seems useful to extract separate features from the peripheral and central parts of the nuclei images

The Prognostic Value of Adaptive Nuclear Texture Features from Patient Gray Level Entropy Matrices in Early Stage Ovarian Cancer

Background: Nuclear texture analysis gives information about the spatial arrangement of the pixel gray levels in a digitized microscopic nuclear image, providing texture features that may be used as quantitative tools for prognosis of human cancer. The aim of the study was to evaluate the prognostic value of adaptive nuclear texture features in early stage ovarian cancer

Stromal composition predicts recurrence of early rectal cancer after local excision

AIMS After local excision of early rectal cancer, definitive lymph node status is not available. An alternative means for accurate assessment of recurrence risk is required to determine the most appropriate subsequent management. Currently used measures are suboptimal. We assess three measures of tumour stromal content to determine their predictive value after local excision in a well-characterised cohort of rectal cancer patients without prior radiotherapy. METHODS AND RESULTS A total of 143 patients were included. Haematoxylin and eosin (H&E) sections were scanned for (i) deep neural network (DNN, a machine-learning algorithm) tumour segmentation into compartments including desmoplastic stroma and inflamed stroma; and (ii) digital assessment of tumour stromal fraction (TSR) and optical DNA ploidy analysis. 3' mRNA sequencing was performed to obtain gene expression data from which stromal and immune scores were calculated using the ESTIMATE method. Full results were available for 139 samples and compared with disease-free survival. All three methods were prognostic. Most strongly predictive was a DNN-determined ratio of desmoplastic to inflamed stroma >5.41 (P 36.5% (P = 0.037). CONCLUSIONS The DNN-determined ratio of desmoplastic to inflamed ratio is a novel and powerful predictor of disease recurrence in locally excised early rectal cancer. It can be assessed on a single H&E section, so could be applied in routine clinical practice to improve the prognostic information available to patients and clinicians to inform the decision concerning further management